Image pickup apparatus and control method thereof

ABSTRACT

In an image pickup apparatus, image data obtained by an image pickup unit is subjected to predetermined image processing, and an image is displayed on an electronic viewfinder screen based on the image data that has undergone the image processing. A specified color is determined based on color information included in a predetermined region of an image that is currently being displayed on the electronic viewfinder screen, and image processing parameters are produced so that color conversion is carried out on the image while eliminating color components of colors other than the determined specified color.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.11/339,960 filed Jan. 26, 2006, which claims priority from JapanesePatent Application No. 2005-023506 filed Jan. 31, 2005, both of whichare hereby incorporated by reference herein in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image pickup apparatus that allows auser to customize a color of an image, and relates to a control methodfor the image pickup apparatus.

2. Description of the Related Art

In recent years, digital cameras have become very popular. As a result,the needs of users with respect to digital cameras have become more andmore diversified.

In order to cope with such a situation, some digital cameras enable auser to customize parameters such as hue, color saturation, lightness,and the like and also enable color reproduction as desired by the userat the time of picking up an image (refer to Japanese Patent ApplicationLaid-Open No. 11-187351). However, it is difficult to show arelationship between variance in the parameters and variance in thecolor of the image, and thus it is necessary for the user to be highlyskilled in performing an appropriate setting of the parameters.

As a proposal related to a method for allowing a user to readily adjustthe color of an image, there is a configuration with which colorconversion processing for converting a specified source color to aspecified target color by specifying a desired source color in an imageduring a retouching process thereof and specifying a color desired asthe target color for conversion is carried out (refer to Japanese PatentApplication Laid-Open No. 2004-129226). However, with thisconfiguration, it is necessary for the user to specify the target colorwith respect to the source color, and, therefore, it is difficult toreadily produce an image with a special image effect in an image pickupapparatus in accordance with an operation for picking up an image.

SUMMARY OF THE INVENTION

The present invention is provided in consideration of the foregoing, andthe present invention is directed to an image pickup apparatus having arestricted user interface that is configured to enable a special effectto be explicitly, definitely, and readily set by a user with respect toan image to be picked up, and also to enable performance of the settingto be implemented with a simple operation at the time of picking up animage by the user.

In one aspect of the present invention, an image pickup apparatusincludes an image pickup unit, an image processing unit configured toprocess an image signal obtained by the image pickup unit, a recordingunit configured to record the image signal that is outputted from theimage processing unit, a display unit configured to display on anelectronic viewfinder screen an image based on the image signal that ispicked up by the image pickup unit and outputted from the imageprocessing unit, a color determination unit configured to determine acolor value based on color information included in a predeterminedregion of an image that is being displayed on the electronic viewfinderscreen, and a color conversion parameter determination unit configuredto determine color conversion parameters for leaving the color valuedetermined by the color determination unit and converting a colorcomponent value included in the image signal other than the color valueinto a predetermined color value, wherein the image signal is processedby the image processing unit based on the parameters determined by thecolor conversion parameter determination unit and the processed image isdisplayed by the display unit.

With this configuration, a user can freely, explicitly, and readily seta specified color even with a restricted user interface of the imagepickup apparatus, and further, it is possible, in picking up an image,to obtain an image that is subjected to a special effect such that acolor conversion by which a color specified by the user is left andcolor component values of colors other than the specified color areconverted into predetermined color component values, with a simpleoperation of the user interface.

In another aspect of the present invention, a method of controlling theimage pickup apparatus, a control program for controlling of the imagepickup apparatus, and a storage medium that stores therein the controlprogram that are in correspondence with the image pickup apparatus areprovided.

Further features of the present invention will become apparent from thefollowing detailed description of exemplary embodiments with referenceto the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will now be described, by way of example only, withreference to the accompanying diagrammatic drawings, in which:

FIG. 1 is a block diagram showing a configuration of an image pickupapparatus according to an embodiment of the present invention.

FIG. 2 is a perspective view showing an external appearance of the imagepickup apparatus according to the embodiment of the present invention.

FIG. 3 is a block diagram showing a method of image processing accordingto an embodiment of the present invention.

FIG. 4 is a flow chart that shows a method of processing to be carriedout in a color leaving mode according to a first embodiment of thepresent invention.

FIG. 5 is a conceptual diagram that shows a color array of a CCD in theimage pickup apparatus according to an embodiment of the presentinvention.

FIG. 6 is a conceptual diagram that shows how data after a CCD signal isinterpolated in the image pickup apparatus according to an embodiment ofthe present invention.

FIG. 7 is a view that shows a filter as used for luminance signalgeneration processing according to an embodiment of the presentinvention.

FIG. 8 is a view showing an example of an electronic viewfinder (EVF)screen in a specified color capturing mode according to an embodiment ofthe present invention.

FIG. 9 is a view that shows a method of color conversion processingusing a three-dimensional lookup table according to an embodiment of thepresent invention.

FIG. 10 is a flow chart that shows a method of processing in a colorleaving mode according to a second embodiment of the present invention.

FIG. 11 is a flow chart that shows a method for performing an exposurecontrol and a white balance control during an operation of a colorleaving mode according to a third embodiment of the present invention.

FIG. 12 shows an image displayed on an EVF screen when a subject isbeing photographed according to an embodiment of the present invention.

FIG. 13 is a view showing an image after being subjected to theprocessing carried out in the color leaving mode according to anembodiment of the present invention.

FIG. 14 is a flow chart that shows an operation at the time of fixing anexposure and an operation at the time of fixing a white balancecoefficient in the color leaving mode according to the first embodimentof the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention will be described indetail in accordance with the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram showing an example of a configuration of animage pickup apparatus 100 (in this embodiment, a digital camera)according to a first embodiment of the present invention. An image inreal space is formed on an image pickup element 103 that converts anoptical image into an electric signal, via an image pickup lens 101 anda shutter 102 that is provided with a diaphragm function. An A/Dconversion section 105 converts an analog signal that is outputted fromthe image pickup element 103 into a digital signal. A timing-generatingsection 106 is controlled by a memory control section 108 and a systemcontrol section 109, and supplies a control signal and clock signals tothe image pickup element 103, the A/D conversion section 105, and a D/Aconversion section 107.

An image processing section 110 carries out a given process, such asimage interpolation processing and color conversion processing, on datafrom the A/D conversion section 105 or data from the memory controlsection 108. The image processing section 110 also carries outcomputation processing using taken image data. The system controlsection 109 controls an exposure control section 111 and a distancemeasurement control section 112 on the basis of the result of thecomputation by the image processing section 110. In addition, the systemcontrol section 109 carries out through-the-lens (TTL) processing suchas an automatic focus (AF) processing, automatic exposure (AE)processing, and preliminary emission of flash (EF) processing. Further,the image processing section 110 carries out computation processingusing taken image data, and also carries out TTL-type automatic whitebalance (AWB) processing on the basis of the result of the computation.

The memory control section 108 controls the A/D conversion section 105,the timing-generating section 106, the D/A conversion section 107, theimage processing section 110, an image display memory 113, a memory 114,and a compression/expansion section 115. The data outputted from the A/Dconversion section 105 is written into the image display memory 113 orthe memory 114 via the image processing section 110 and the memorycontrol section 108, or via the memory control section 108 only. Notethat in writing image data into the image display memory 113, the imagedata is written while being thinned in accordance with the resolution ofa display device of the image display section 116. The image data usedfor display that is written into the image display memory 113 isconverted into an analog signal for image display via the D/A conversionsection 107, and is then displayed by the image display section 116. Theimage display section 116 is configured by a thin-film transistor liquidcrystal display (TFTLCD) and the like. Note that by sequentiallydisplaying data of taken images by using the image display section 116,it is possible to implement a so-called electronic viewfinder function.The image display section 116 is capable of arbitrarily turning on oroff a display in accordance with an instruction from the system controlsection 109. If the display is turned off, power consumption of theimage pickup apparatus 100 can be significantly reduced.

The memory 114 is a memory for storing therein still images and movingimages that are picked up. The memory 114 has storage capacity largeenough to store a given number of still images and a given length oftime of moving images. Thereby it is possible to write a large number ofimages at a high speed into the memory 114 even in the case ofcontinuous shooting in which a plurality of still images arecontinuously taken and also in the case of panoramic shooting. Thememory 114 can also be used as a work area of the system control section109.

The compression/expansion section 115 compresses and expands image databy using an adaptive discrete cosine transform (ADCT) or the like. Thecompression/expansion section 115 reads an image stored in the memory114, compresses or expands the read image, and writes data of theprocessed image into the memory 114.

The exposure control section 111 controls the shutter 102 having adiaphragm function. The exposure control section 111 is also incoordination with the flash 117 to effect a flash light intensitycontrol function. The distance measurement control section 112 controlsfocusing of the image pickup lens 101. A zoom control section 118controls zooming of the image pickup lens 101. A barrier control section119 controls an operation of a protection section 151. The protectionsection 151 is a barrier that prevents an image pickup section frombeing smeared or damaged by covering the image pickup section of theimage pickup apparatus 100 that includes the image pickup lens 101, theshutter 102, and the image pickup element 103. The protection section151 is primarily intended, in general, to protect the image pickup lens101. The flash 117 is provided with an AF auxiliary light projectingfunction and a flash light intensity control function. The exposurecontrol section 111 and the distance measurement control section 112 arecontrolled by the TTL processing. That is, the system control section109 controls the exposure control section 111 and the distancemeasurement control section 112 on the basis of the result ofcomputation obtained from taken image data by the image processingsection 110. The system control section 109 controls the entirety of theimage pickup apparatus 100. The memory 120 stores therein constants,variables, programs, and the like for operation of the system controlsection 109. Programs described in FIGS. 4, 10, 11 and 14 are stored inmemory 120, and they are executed by image processing section 110 and/orsystem control section 109.

The indicator section 121 is a liquid crystal display (LCD), alight-emitting diode (LED), or the like that indicates a state ofoperation, a message, and the like with characters or images, inaccordance with execution of a program by the system control section109. Note that the indicator section 121 may include a speaker, apiezoelectric buzzer (sound production device), and the like capable ofoutputting a voice, a buzzer sound, and the like that expresses orrepresents the state of operation or a part of the message. Further, theindicator section 121 can be disposed at one position or two or morepositions that are easy to look at in the vicinity of an operationsection of the image pickup apparatus 100. Some functions and mechanismsof the indicator section 121 may be installed within an optical finder104.

Contents displayed on the indicator section 121 to be displayed on theLCD and the like include a single/continuous shooting display, aself-timer display, a compression ratio display, a recording pixelnumber display, a recording image number display, a remainingphotographable image number display, a shutter speed display, anaperture value display, an exposure compensation display, a flashdisplay, a red-eye reduction display, a macrophotography display, abuzzer setting display, a clock battery remaining quantity display, abattery remaining quantity display, an error display, an informationdisplay by multiple-digit, a display of attachment or detachment of arecording medium 122 and a recording medium 123, a communication I/Foperation display, a date and time display, and the like. Further,contents displayed on the indicator section 121 to be displayed withinthe optical finder 104 include a focusing display, a camera-shakewarning display, a flash charge display, a shutter speed display, anaperture value display, a exposure compensation display, and the like.

The non-volatile memory 124 can be a memory that can be electricallyerased and recorded. For the non-volatile memory 124, an EEPROM or thelike, for example, is used. Reference numerals 125, 126, 127, 128, and129 respectively denote an operation section for inputting various kindsof operational instructions to the system control section 109. Theoperation sections are configured by any one of, or a combination of anyones of: a switch, a dial, a touch screen, pointing by detection of aline of sight, a speech recognition device, and the like. The operationsections are specifically explained as described below.

A mode dial switch 125 is a switch for switching and setting eachfunction mode, such as power off, an automatic photographing mode, amanual photographing mode, a panoramic photographing mode, areproduction mode, a multiscreen reproduction/erasure mode, a PCconnection mode, and the like. A shutter switch 126 outputs a signal SW1when a shutter button (reference numeral 203 in FIG. 2) is operated(when the shutter button is half-pressed) and outputs a signal SW2 whenthe operation of the shutter button is completed (when the shutterbutton is fully pressed). Start of operation of processing, such asautomatic focus (AF) processing, automatic exposure (AE) processing,automatic white balance (AWB) processing, and preliminary emission offlash (EF) processing, is instructed by the signal SW1. The start of aseries of photographing processing operations is instructed by thesignal SW2. In the photographing processing, a series of processingoperations as described below is carried out. That is, the signal readfrom the image pickup element 103 is converted into a digital signal(subjected to exposure processing) by the A/D conversion section 105 andthe converted signal is written into the memory 114 via the memorycontrol section 108 as image data (RAW data). Then, the signal that isexposure processed is subjected to a computation processing (developingprocessing) by the image processing section 110 and the memory controlsection 108, and is then written into the memory 114. Further, the imagedata written into the memory 114 is read from the memory 114, and iscompressed by the compression/expansion section 115, and then is writtenonto the recording medium 122 or the recording medium 123 (recordingprocessing).

An image display on/off switch 127 switches between and sets on/offstates of the image display section 116. With this function, in takingan image by using the optical finder 104, it is possible to cut offpower supply to the image display section 116 configured by a TFTLCD andthe like, and thereby it is possible to effect power saving. A quickreview on/off switch 128 sets on/off of a quick review function thatautomatically reproduces photographed image data immediately after theimage is photographed. Note that the quick review on/off switch 128 isprovided with a function for setting the quick review function forquickly reviewing a photographed image in a case where the image displaysection 116 is turned off, namely, a function for enabling reviewing aphotographed image even in a case where image display is off).

An operation section 129 is provided with various kinds of buttons, atouch screen, and the like, and functions as an instruction button forperforming various kinds of operations, by one single switch or by acombination of a plurality of switches. For the operational instructionbutton like this, there are, for example, a menu button, a settingbutton, a macro button, a multiscreen reproduction page-turning button,a flash setting button, a single-shooting/continuous-shooting/self-timerswitching button, a menu shifting “+” button, a menu shifting “−”button, a reproduction image shifting “+” button, a reproduction imageshifting “−” button, a photographing image quality selection button, anexposure compensation button, a date/time setting button, an imageerasure button, an image erasure cancel button, and the like.

A power control section 131 is configured by a battery detectioncircuit, a DC-DC converter, a switch circuit for switching betweenblocks to be energized, and the like. The power control section 131determines whether a battery cell is mounted or not, what type themounted battery cell is, and how much charge is remaining in the batterycell. The power control section 131 also controls the DC-DC converter inaccordance with the result of the determination and the instruction fromthe system control section 109. Further, the power control section 131supplies necessary voltage to each section, including a recordingmedium, for a necessary period of time. A power supply section 134 isconfigured by a primary battery such as an alkaline battery, a lithiumcell, and the like, a secondary battery such as a NiCd battery, a NiMHbattery, a Li battery, and the like, an AC adapter, and the like. Thepower supply section 134 is connected to the power control section 131via connectors 132 and 133.

An interface 135 and an interface 136 respectively connect recordingmediums 122 and 123, such as a memory card or a hard disk, with a bus inthe image pickup apparatus 100. The interface 135 and the interface 136are connected to the recording mediums 122 and 123 via connectors 137and 138, respectively. The recording medium attachment/detachmentdetection section 130 determines whether or not the recording medium 122and/or the recording medium 123 is connected to the connector 137 and/orthe connector 138.

Note that in this embodiment, an explanation is made as to a case wheretwo systems of interfaces and connectors to which a recording medium ismounted or connected are provided. However, the configuration of thisembodiment may be such that the number of interfaces and connectors towhich a recording medium is mounted or connected may be either one ormore than one. Further, the configuration may be such that interfacesand connectors of different standards are provided in combination. Forthe interface and the connector, those in compliance with the standardof a PCMCIA card, a CompactFlash (CF) (registered trademark) card, andthe like may be used.

Further, in a case where the interface 135, the interface 136, theconnector 137, and the connector 138 that are in compliance with thestandard of the PCMCIA card, the CompactFlash (CF) (registeredtrademark) card, and the like are used, the configuration may be suchthat image data and management information adjunct to the image data maybe sent/transferred and received between the image pickup apparatus ofthis embodiment and other computers and/or a peripheral device such as aprinter and the like, by connecting to the image pickup apparatus ofthis embodiment various kinds of communication cards of standards, suchas a LAN card, a modem card, a USB card, an IEEE 1394 card, a P1284card, a SCSI card, a communication card for PHS, and the like.

The optical finder 104 enables photographing by using an optical finderonly, without using an electronic viewfinder function by the imagedisplay section 116. Besides, as described above, within the opticalfinder 104, there is provided a certain function of the indicatorsection 121, such as, for example, a focusing display, a camera-shakewarning display, a flash charge display, a shutter speed display, anaperture value display, an exposure compensation display, and the like.

The communication section 145 is provided with various communicationfunctions in compliance with a standard, such as RS232C, USB, IEEE1394,P1284, SCSI, modem, LAN, wireless communication, and the like. Theconnector 146 is a connector for connecting the image pickup apparatus100 to other devices via the communication section 145. Otherwise, inthe case of wireless communication, the connector 146 is an antenna.

The recording medium 122 and the recording medium 123 are provided,respectively, with recording sections 139 and 142, such as asemiconductor memory or a magnetic disc, interfaces 140 and 143 for usewith the image pickup apparatus 100, and connectors 141 and 144 forconnection to the image pickup apparatus 100.

FIG. 2 is a perspective view of the image pickup apparatus (digitalcamera) 100 of this embodiment. A power switch 201 is a button forturning on and off the power supply. Reference numeral 202 and referencenumerals 205 through 209 denote elements constituting one part of theoperation section 129 as described above. A mode switching lever 202 isa lever for switching and setting each function mode, such as aphotographing mode, a reproduction mode, a moving image photographingmode, a still image photographing mode, and the like. A shutter button203 is a button that functions as the shutter switch 126 as describedabove. An LCD 204 configures one part of the image display section 116as described above and functions as an electronic viewfinder, and alsodisplays a screen obtained by reproducing still images and/or movingimages. A menu button 205 is a switch that turns on and off a menuscreen for changing photographing parameters and the setting of thecamera. A setting button 206 is a button used for selecting anddetermining a menu displayed on the menu screen by the operation of themenu button 205. A deletion button 207 is a button for designating thedeletion of an image. A display button 208 configures the image displayon/off switch 127, and is a button for switching between display andnondisplay on the LCD 204. Across-shaped button 209 is a button that canbe used for shifting between items displayed on the menu screen by usingthe cross-shaped upper/lower/left/right buttons and for feeding an imageby pressing the left/right buttons in a reproduction mode.

FIG. 3 is a block diagram that explains processing of the imageprocessing section 110 in the digital camera 100 according to thisembodiment. Note that parameters as used for each processing (parametersfor a matrix operation and a three-dimensional lookup table) are storedin the memory 120, and are read when required by the image processingsection 110. A CCD digital signal subjected to A/D conversion by the A/Dconversion section 105 is then subjected to white balance processing bya white balance processing section 301. Although no explanation is madeas to the white balance processing here, the white balance processingcan be carried out by using a method as described in Japanese PatentApplication Laid-Open No. 2003-244723, for example. The CCD digitalsignal that has been subjected to the white balance processing issupplied to an interpolation processing section 302. Note that the imagepickup element 103 of this embodiment has a Bayer array color filter asshown in FIG. 5. Accordingly, in the interpolation processing section302, the processing for converting Bayer array CCD data shown in FIG. 5into interpolated data of R, G1, G2, and B as shown in FIG. 6 is carriedout. The interpolated CCD digital signals are inputted to a matrixoperation processing section 303. In the matrix operation processingsection 303, a 4×3 matrix operation as expressed by equation (1) isperformed to obtain values Rm, Gm, and Bm.

$\begin{matrix}{{\begin{matrix}\begin{matrix}{Rm} \\{Gm}\end{matrix} \\{Bm}\end{matrix}} = {{\begin{matrix}{M\; 11} & {M\; 21} & {M\; 31} & {M\; 41} \\{M\; 12} & {M\; 22} & {M\; 32} & {M\; 42} \\{M\; 13} & {M\; 23} & {M\; 33} & {M\; 43}\end{matrix}}{\begin{matrix}\begin{matrix}R \\{G\; 1}\end{matrix} \\\begin{matrix}{G\; 2} \\B\end{matrix}\end{matrix}}}} & (1)\end{matrix}$

The CCD digital signals that have undergone the matrix operationprocessing are then amplified by a gain by a color difference gainoperation processing section 304, and thereby converted intocolor-difference signals. In other words, the signals Rm, Gm, and Bm areconverted into Y, Cr, and Cb signals in accordance with the computationas expressed by equation (2) below. Then, the resulting Cr and Cbsignals are amplified by a gain in accordance with the arithmeticoperation processing as expressed by equation (3) below. After that, bythe computation as expressed by equation (4) (inverse matrix operationof equation (2)), the signals are converted into Rg, Gg, and Bg signals.

$\begin{matrix}{{\begin{matrix}\begin{matrix}Y \\{Cr}\end{matrix} \\{Cb}\end{matrix}} = {{\begin{matrix}0.3 & 0.59 & 0.11 \\0.7 & {- 0.59} & {- 0.11} \\{- 0.3} & {- 0.59} & 0.89\end{matrix}}{\begin{matrix}\begin{matrix}{Rm} \\{Gm}\end{matrix} \\{Bm}\end{matrix}}}} & (2) \\{{{Cr}^{*} = {G\; 1 \times {Cr}}}{{Cb}^{*} = {G\; 1 \times {Cb}}}} & (3) \\{{\begin{matrix}\begin{matrix}{Rg} \\{Gg}\end{matrix} \\{Bg}\end{matrix}} = {{\begin{matrix}0.3 & 0.59 & 0.11 \\0.7 & {- 0.59} & {- 0.11} \\{- 0.3} & {- 0.59} & 0.89\end{matrix}}^{- 1}{\begin{matrix}\begin{matrix}Y \\{Cr}^{*}\end{matrix} \\{Cb}^{*}\end{matrix}}}} & (4)\end{matrix}$

The CCD digital signals that have undergone a color difference gainoperation processing are sent to a gamma processing section 305. Thegamma processing section 305 performs gamma conversion processing of theCCD digital signals using equations (5) to (7) below:Rt=GammaTable[Rg]  (5)Gt=GammaTable[Gg]  (6)Bt=GammaTable[Bg]  (7)

where GammaTable represents a one-dimensional lookup table.

The CCD digital signals that are subjected to the gamma conversionprocessing are then sent to a hue correction operation processingsection 306. The hue correction operation processing section 306converts Rt, Gt, and Bt signals into Y, Cr, and Cb signals usingequation (8) as described below. Further, the hue correction operationprocessing section 306 corrects Cr and Cb signals using equation (9),and after that, converts the Y, Cr′, and Cb′ signals into Rh, Gh, and Bhsignals using equation (10) (inverse matrix operation of equation (8)).

$\begin{matrix}{{\begin{matrix}\begin{matrix}Y \\{Cr}\end{matrix} \\{Cb}\end{matrix}} = {{\begin{matrix}0.3 & 0.59 & 0.11 \\0.7 & {- 0.59} & {- 0.11} \\{- 0.3} & {- 0.59} & 0.89\end{matrix}}{\begin{matrix}\begin{matrix}{Rt} \\{Gt}\end{matrix} \\{Bt}\end{matrix}}}} & (8) \\{{\begin{matrix}{Cr}^{*} \\{Cb}^{*}\end{matrix}} = {{\begin{matrix}{H\; 11} & {H\; 21} \\{H\; 12} & {H\; 22}\end{matrix}}{\begin{matrix}{Cr} \\{Cb}\end{matrix}}}} & (9) \\{{\begin{matrix}\begin{matrix}{Rh} \\{Gh}\end{matrix} \\{Bh}\end{matrix}} = {{\begin{matrix}0.3 & 0.59 & 0.11 \\0.7 & {- 0.59} & {- 0.11} \\{- 0.3} & {- 0.59} & 0.89\end{matrix}}^{- 1}{\begin{matrix}\begin{matrix}Y \\{Cr}^{*}\end{matrix} \\{Cb}^{*}\end{matrix}}}} & (10)\end{matrix}$

The CCD digital signals that have been subjected to the processing bythe hue correction operation processing section 306 are then sent to acolor difference signal conversion processing section 307. The colordifference signal conversion processing section 307 generates U and Vsignals from Rh, Gh, and Bh signals by computation as expressed byequation (11).

$\begin{matrix}{{\begin{matrix}U \\V\end{matrix}} = {{\begin{matrix}{- 0.169} & \begin{matrix}{- 0.333} & 0.502\end{matrix} \\0.499 & \begin{matrix}{- 0.421} & {- 0.078}\end{matrix}\end{matrix}}{\begin{matrix}{Rh} \\\begin{matrix}{Gh} \\{Bh}\end{matrix}\end{matrix}}}} & (11)\end{matrix}$

Meanwhile, the CCD digital signals subjected to the white balanceprocessing by the white balance processing section 301 are also suppliedto a luminance signal generation processing section 308. The luminancesignal generation processing section 308 converts the CCD digitalsignals into a luminance signal. For example, a luminance signal in thecase of a primary color filter as shown in FIG. 5 is generated by makingboth R and B signals zero and by carrying out a two-dimensionalfiltering processing having coefficients as shown in FIG. 7. Note that,in the case of a complementary color filter, a signal that is subjectedto the two-dimensional filtering processing having coefficients as shownin FIG. 7 is used as it is as a luminance signal. The luminance signalgenerated by the luminance signal generation processing section 308 isthen subjected to edge enhancement processing by a high frequencyemphasis processing section 309, and is further subjected to gammaconversion processing by a gamma processing section 310, so that a Ysignal is generated.

The Y signal outputted from the gamma processing section 310 and the Uand V signals outputted from the color difference signal conversionprocessing section 307 are converted into Y′, U′, and V′ signals,respectively, by a color conversion processing section 311. In the colorconversion processing section 311, a conversion processing using athree-dimensional lookup table is carried out. With regard to thisprocessing, an explanation is made in detail below.

The digital camera (image pickup apparatus 100) of this embodiment isprovided with an image pickup mode in which a color conversion can beperformed while leaving an arbitrary color specified by a user(hereinafter, this image pickup mode is referred to as a “color leavingmode”). In the color leaving mode, an electronic viewfinder (EVF) screen801 as shown in FIG. 8 is displayed on the LCD 204, a given operation isperformed so that a desired color is included in a color capture frame802 within a photographed image being displayed in real time, andthereby the color of an image within the color capture frame 802 isdetermined as a specified color. More specifically, for example, if theuser specifies an object A that is present in a central portion of theimage by the color capture frame 802 as shown in FIG. 12, the color ofthe object A is determined as a specified color. Here, in this case, theobject A and an object B are in the same color. Thus, a lookup table ofthe color conversion processing section 311 is set so that only thedetermined specified color is left and the other color components areerased. As a result, a displayed image displayed on the EVF screen 801thereafter and a photographed image recorded by operation of the shutterbutton 203 are subjected to conversion so as to be a chromaticallycolored by eliminating the color components of objects other than theobjects A and B and leaving the color of the objects A and B only thatis colored with the specified color, as shown in FIG. 13. Hereinbelow,the color leaving mode according to this embodiment is explained indetail.

First, an explanation is made as to a color leaving processing in thecolor leaving mode. In the color conversion processing section 311, YUVsignals are converted into Y′U′V′ signals by the three-dimensionallookup table. In this embodiment, in order to reduce a capacity of thethree-dimensional lookup table, a signal value range between a minimumvalue and a maximum value of each of the Y signal, U signal, and Vsignal is divided into eight parts to prepare a list of Y, U, and Vsignals (lookup table) corresponding to 9×9×9=729 three-dimensionalrepresentative lattice points. Y, U, and V signals other than therepresentative lattice points are obtained by interpolation. FIG. 9 is aview conceptually showing the three-dimensional lookup table accordingto this embodiment. Each lattice point is assigned with the values ofthe converted Y, U, and V signals. For example, a lattice point 1101 isa point having values (32, 255, 32), and if there arises no differenceafter conversion of the signal, then the lattice point 1101 is assignedwith the values (32, 255, 32). Alternatively, if the lattice point 1101is a point having values (32, 230, 28) after the conversion of thesignal, then the lattice point 1101 is assigned with the values (32,230, 28).

For example, the values of Y, U, and V signals of a point 1103 that ispresent within a cubic lattice 1102 in FIG. 9 are computed byinterpolation arithmetic operation from the Y, U, and V values of eachof the lattice points “a” through “h”, each of which corresponds to eachof the vertices of the cubic lattice 1102. The interpolation arithmeticoperation is performed using equations (12) through (14) as describedbelow. Here, in the equations (12) through (14), let Y, U, and V be theinput Y, U, and V signals, and let Yout (Y, U, V), Uout (Y, U, V), andVout (Y, U, V) be the output Y, U, and V signals. Further, let Yi, Ui,and Vi be the signals at a representative lattice point (“a” in FIG. 9)having values which are smaller than and closest to the signal values ofthe input Y, U, and V signals, and let Yout (Yi, Ui, Vi), Uout (Yi, Ui,Vi), and Vout (Yi, Ui, Vi) be the representative lattice point outputsignals. Further, let Step (in this embodiment, 32) be the step width ofthe representative lattice points. Thus, a lattice point “b”, forexample, is assigned with values (Yi+Step, Ui, Vi), and a lattice point“c” is assigned with values (Yi, Ui+Step, Vi).Y=Yi+YfU=Ui+UfV=Vi+VfYout(Y,U,V)=Yout(Yi+Yf,Ui+Uf,Vi+Vf)=(Yout(Yi,Ui,Vi)×(Step−Yf)×(Step−Uf)×(Step−Vf)+Yout(Yi+Step,Ui,Vi)×(Yf)×(Step−Uf)×(Step−Vf)+Yout(Yi,Ui+Step,Vi)×(Step−Yf)×(Uf)×(Step−Vf)+Yout(Yi,Ui,Vi+Step)×(Step−Yf)×(Step−Uf)×(Vf)+Yout(Yi+Step,Ui+Step,Vi)×(Yf)×(Uf)×(Step−Vf)+Yout(Yi+Step,Ui,Vi+Step)×(Yf)×(Step−Uf)×(Vf)+Yout(Yi,Ui+Step,Vi+Step)×(Step−Yf)×(Uf)×(Vf)+Yout(Yi+Step,Ui+Step,Vi+Step)×(Yf)×(Uf)×(Vf)/(Step×Step×Step)  (12)Uout(Y,U,V)=Uout(Yi+Yf,Ui+Uf,Vi+Vf)=(Uout(Yi,Ui,Vi)×(Step−Yf)×(Step−Uf)×(Step−Vf)+Uout(Yi+Step,Ui,Vi)×(Yf)×(Step−Uf)×(Step−Vf)+Uout(Yi,Ui+Step,Vi)×(Step−Yf)×(Uf)×(Step−Vf)+Uout(Yi,Ui,Vi+Step)×(Step−Yf)×(Step−Uf)×(Vf)+Uout(Yi+Step,Ui+Step,Vi)×(Yf)×(Uf)×(Step−Vf)+Uout(Yi+Step,Ui,Vi+Step)×(Yf)×(Step−Uf)×(Vf)+Uout(Yi,Ui+Step,Vi+Step)×(Step−Yf)×(Uf)×(Vf)+Uout(Yi+Step,Ui+Step,Vi+Step)×(Yf)×(Uf)×(Vf)/(Step×Step×Step)  (13)Vout(Y,U,V)=Vout(Yi+Yf,Ui+Uf,Vi+Vf)=(Vout(Yi,Ui,Vi)×(Step−Yf)×(Step−Uf)×(Step−Vf)+Vout(Yi+Step,Ui,Vi)×(Yf)×(Step−Uf)×(Step−Vf)+Vout(Yi,Ui+Step,Vi)×(Step−Yf)×(Uf)×(Step−Vf)+Vout(Yi,Ui,Vi+Step)×(Step−Yf)×(Step−Uf)×(Vf)+Vout(Yi+Step,Ui+Step,Vi)×(Yf)×(Uf)×(Step−Vf)+Vout(Yi+Step,Ui,Vi+Step)×(Yf)×(Step−Uf)×(Vf)+Vout(Yi,Ui+Step,Vi+Step)×(Step−Yf)×(Uf)×(Vf)+Vout(Yi+Step,Ui+Step,Vi+Step)×(Yf)×(Uf)×(Vf)/(Step×Step×Step)  (14)

Hereinbelow, the equation for the lookup table conversion and theinterpolation arithmetic operation equation of the equations (12), (13),and (14) are simply expressed as equation (15) as described below. Here,in the equation (15), Y, U, and V express the input signal values, LUTdenotes a 9×9×9 lookup table as shown in FIG. 9, and Yout, Uout, andVout express the results of the lookup table conversion and theinterpolation arithmetic operation (Y′, U′, and V′ in FIG. 3). That is,the color conversion processing section 311 carries out the conversionprocessing as expressed by equation (15) as described below.(Yout,Uout,Vout)=LUT[(Y,U,V)]  (15)

As described above, when a specified color is determined in the colorleaving mode, a cubic lattice that includes the specified color thereinis determined, and then the values of each of the lattice points thatform the cubic lattice are changed, without changing the Y, U, and Vsignal values of a coordinate position of the specified color, so thatthe U and V signals of other coordinate positions are set to be zero.For example, if the specified color determined by the processing asdescribed with reference to FIG. 9 has Y, U, and V values of the point1103, while the values of the lattice points “a” through “h” of thecubic lattice 1102 are not changed, the values of the representativepoints other than the lattice points “a” through “h” of the cubiclattice 1102 are changed so that the U and V signal values are set to bezero, which indicates an achromatic color, without changing the Ysignal. That is, all the colors other than the specified color arechanged into a gray scale. Then, in the color conversion processingsection 311, the color conversion processing is performed by using thethree-dimensional lookup table that has undergone the change of thevalues. Note that, in the following discussion, the change of the valuesof the lattice points as described above is referred to as “setting ofparameters”.

As described above, lattice point data of the three-dimensional lookuptable is determined in accordance with the specified color to performthe color conversion, and thereby it is possible to readily perform acolor setting as favored and desired by the user on an image to bereproduced. In addition, in the color conversion processing, only therepresentative lattice points in the vicinity of the color that isdesired to be changed are changed in the three-dimensional lookup table.Therefore, it is possible to readily implement the conversion such thatthe color of one part of an image is left and the color signals of theother colors of the image are eliminated, with a high speed operation.That is, the parameters that are utilized by the matrix operationprocessing section 303, the color difference gain operation processingsection 304, the gamma processing section 305, the hue correctionoperation processing section 306, and the like are not changed, andthereby it is possible to leave only the desired color (a region of thecolor including the specified color).

FIG. 4 is a flow chart that explains processing operation of the digitalcamera of this embodiment at the time of photographing in the colorleaving mode. The memory 120 stores therein the programs for executingthe flow chart in FIG. 4, and the system control section 109 executesthe operation of the programs. Note that there is no difference ofoperations carried out in the photographing mode other than the colorleaving mode from the operations in a common digital camera, andaccordingly, the explanation is made only as to the color leaving modehereinbelow.

When a user sets the photographing mode of the digital camera to thecolor leaving mode, in step S401, previous setting parameters that havebeen set in the previous color leaving mode are set as the parameters ofthe color conversion processing section 311. In step S402, the systemcontrol section 109 determines whether an exposure control start timinghas come. If it is determined that the exposure control start timing hascome, the exposure control section 111 carries out exposure processingin step S403. The exposure processing is the setting of exposure fordisplaying an image on the EVF screen. Frequent performance of theexposure processing causes flickering of an image displayed on thescreen, and therefore a time constant is set so that the exposureprocessing is set to be performed at a certain time interval. Forexample, the exposure processing is performed once in every two seconds.Accordingly, the result of the determination in step S402 becomespositive at this interval, and then, exposure control is carried out instep S403.

Next, in step S404, the system control section 109 determines whether awhite balance control start timing has come. If it is determined thatthe white balance control start timing has come, processing proceeds tostep S405. In step S405, white balance processing is performed. Just asin the case of exposure processing, frequent performance of the whitebalance processing causes flickering of an image displayed on thescreen, and therefore a time constant is set so that the white balanceprocessing is performed at a certain time interval. For example, thewhite balance processing is set to be performed once in every fiveseconds. In the white balance processing, a white balance coefficientfor performing the white balance processing is computed, and the whitebalance coefficient used by the image processing section 110 is updated.

In step S406, an image is photographed with an aperture value set by theexposure control in step S403, and the image processing section 110carries out image processing on a through image that is a real-timeoutput from the image pickup element 103 by using the white balancecoefficient set in step S405. Then, in step S407, the image data thathas undergone the image processing in step S406 is displayed on the LCD204 (the image display section 116) that functions as the EVF screen.

The EVF screen 801 as shown in FIG. 8 is displayed on the LCD 204. Asshown in FIG. 8, in the color leaving mode, the EVF screen 801, a colorcapture frame 802, and a specified color display frame 803, which arelocated within the EVF screen 801, are displayed on the LCD 204. Thus,it is possible to perform setting of the specified color by performinggiven operations of the operation section 129 (steps S408 through S410)and to take an image by operation of the shutter button 203 (steps S412and S413).

In this regard, first, an explanation is made as to how the specifiedcolor is set. The user, in order to specify the specified color,determines the direction of the camera, operates an optical zooming, andsets a field angle so that the desired color is fully included in thecolor capture frame 802. When the left button of the cross-shaped button209 is pressed, it is determined that an instruction for capturing thespecified color is inputted, and then processing proceeds from step S408to step S409. In step S409, pixel data of the image that is currentlypresent within the color capture frame 802 is obtained. In step S410,arithmetic operation is carried out to average the obtained pixel data,and the obtained image data that is averaged is determined as thespecified color. When the specified color is determined, a patchindicating the specified color is displayed in the specified colordisplay frame 803.

Note that in computing the average pixel value within the color captureframe 802 in step 410, the pixel data used therein may be image datathat is subjected to thinning for display on the electronic viewfinder(the image data stored in the image display memory 113), or may be imagedata that is stored in the memory 114.

When the specified color is determined in step S410, processing proceedsto step S411. In step S411, conversion parameters for leaving thespecified color only and eliminating the other color signals aredetermined. In this embodiment, as described with reference to FIG. 9and the like above, the parameters are determined so that the signalvalues of the lattice points that form the cubic lattice including thespecified color in the three-dimensional lookup table are not changed,and all the U and V signals of the other lattice points are set to bezero. That is, a luminance Y is not changed, as shown in FIG. 13, withrespect to the colors other than the specified color and the color ofthe values of the lattice points included within the cubic lattice thatincludes therein the specified color, and consequently, the image datais reproduced in a monochromatic tone except the specified color. Then,in step S411, the three-dimensional lookup table of the color conversionprocessing section 311 is updated. Thus, when processing returns to stepS402 to perform an ordinary photographing operation, the updated lookuptable is applied to images captured thereafter from the image pickupelement 103 (step S406), and the image display for the EVF screen iscarried out (step S407). Then, when an image is ordinarily photographed,the three-dimensional lookup table that is updated by the colorconversion processing section 311 is used in the image processing by theimage processing section 110 in photographing an image (steps S412 andS413). Note that in photographing an image, as described above, thesignal SW1 is produced in a state where the shutter button 203 ishalf-pressed, then the processing operations for image taking, such asthe automatic focus (AF) processing, the automatic exposure (AE)processing, the automatic white balance (AWB) processing, thepreliminary emission of flash (EF) processing, and the like, areperformed. Then, the user fully presses the shutter button 203, and inthis state, the signal SW2 is produced. Thereby, a series of imagetaking processing operations is carried out. With the series of imagetaking processing operations, image data that has undergone the imageprocessing, in which the specified color thereof that has undergone theimage processing by the image processing section 110 is left and theother colors thereof are changed into a monochromatic tone reproduction(gray scale), is subjected to the compression processing by thecompression/expansion section 115, and then is recorded on the recordingmedium 122 or the recording medium 123. In the color leaving mode, itmay control only to take a picture of the color leaving after thedesignated color is decided. Hereby, the color leaving mode operation ismade easier.

Note that even in the color leaving mode, it is possible to perform anordinary image taking operation unless the specified color is capturedand the parameters for the color leaving mode are not produced, andthus, the user does not miss a moment for a good image.

Further, the three-dimensional lookup table for use in the color leavingmode that has been produced is stored in the non-volatile memory 124,and thus, the produced three-dimensional lookup table is effective inthe next photographing occasion even after the power of the camera isturned off. Therefore, it is possible to continue shooting with the samecolor leaving effect even if the setting is not performed again usingthe color leaving mode.

Note that in this embodiment, an explanation is made as to an example ofthe case where only one specified color is set. However, theconfiguration of this embodiment is not limited to this. That is, theconfiguration may be such that a plurality of colors desired to be leftcan be set at the same time.

In addition, in this embodiment, the left button of the cross-shapedbutton 209 is used in capturing a specified color. However, theconfiguration of this embodiment is not limited to this. That is, thefunction for capturing a specified color may be assigned to any otherbutton, or a dedicated button may be provided, as a matter of course.

In this embodiment, the color capture frame displayed on the EVF screenwhen the specified color is being captured is fixed in the vicinity ofthe central portion of the screen. However, the color capture frame maybe configured to be able to move to an arbitrary position within the EVFscreen, as freely instructed by the user. In addition or alternatively,the color capture frame may be configured so that the size thereof canbe varied as freely instructed by the user.

Also, in this embodiment, the three-dimensional lookup table processingand the interpolation arithmetic operation processing are used forarithmetic operation processing by the color conversion processingsection 311. However, the configuration of this embodiment is notlimited to this. That is, as long as the three-dimensional lookup tableis supplied with all the necessary signal values, the interpolationarithmetic operation processing is not necessary. In addition, if amatrix arithmetic operation processing is a processing that is capableof setting the color signals of the colors other than the specifiedcolor to be zero, the matrix arithmetic operation processing in whichcoefficients for matrix arithmetic operation are changed for each colorspace, for example, may be used.

Here, processing using the matrix arithmetic operation processing isbriefly explained. In the above-described embodiment, the values of Y,U, and V signals after being subjected to conversion are set on each ofthe lattice points as shown in FIG. 9. However, in the processing usingthe matrix arithmetic operation processing, coefficients M11 through M33of equation (16) as described below are stored in each of the latticepoints. The coefficients M11 through M33 are determined in accordancewith signals Yin, Uin, and Vin. Then, the arithmetic operation asexpressed by equation (16) is carried out to obtain signals Yout, Uout,and Vout. Here, the coefficients M11 through M33 may be obtained anddetermined in accordance with the coefficients stored in the latticepoint that is located at a position closest to the signals Yin, Uin, andVin, or in accordance with the result of interpolation arithmeticoperation in relation to each of the lattice points, as in theabove-described embodiment.

$\begin{matrix}{{\begin{matrix}\begin{matrix}{Yout} \\{Uout}\end{matrix} \\{Vout}\end{matrix}} = {{\begin{matrix}{M\; 11} & \begin{matrix}{M\; 12} & {M\; 13}\end{matrix} \\\begin{matrix}{M\; 21} \\{M\; 31}\end{matrix} & \begin{matrix}\begin{matrix}{M\; 22} \\{M\; 32}\end{matrix} & \begin{matrix}{M\; 23} \\{M\; 33}\end{matrix}\end{matrix}\end{matrix}}{\begin{matrix}{Yin} \\\begin{matrix}{Uin} \\{Vin}\end{matrix}\end{matrix}}}} & (16)\end{matrix}$

Second Embodiment

In the first embodiment of the present invention, the exposureprocessing and the white balance processing are, respectively, simplycarried out at time intervals determined by the given time constants(steps S402 through S405). In the second embodiment, at the time ofcapturing an image for determination of a specified color (step S409),the white balance processing and the exposure control processing thatare different from those in the case of the ordinary image takingoperation are performed. Thus, the exposure and the white balance at thetime of capturing a specified color are more appropriately controlled,so that capturing of the specified color with higher accuracy isimplemented.

Hereinbelow, a configuration of the second embodiment that is differentfrom the configuration of the first embodiment is explained. In thesecond embodiment, in the color capturing processing in step S409 asdescribed in the first embodiment, the exposure control processing andthe white balance processing are carried out. FIG. 10 is a flow chartthat explains the operation of capturing image information of an imagewithin the color capture frame 802 according to the second embodiment.FIG. 10 shows processing equivalent to the processing in step S409 asshown in FIG. 4, as described with respect to the first embodiment.Hereinbelow, a detailed explanation is made as to the second embodimentwith reference to FIG. 10.

When the left button of the cross-shaped button 209 is pressed, it isdetermined that an instruction for capturing a specified color isinputted, and then processing proceeds from step S408 to step S409. Instep S409, processing operations shown in steps S501 through S504 arecarried out.

In this regard, first, the same exposure control processing as theprocessing in step S403 is carried out in step S501. In step S502, thesame white balance processing as the processing in step S405 is carriedout. In the exposure control processing, an exposure is determined, andthen, an aperture value and a shutter speed are determined on the basisof the determined exposure (step S501). The image pickup apparatuscarries out image taking with the determined aperture value and shutterspeed, and also carries out the white balance processing on the basis ofthe photographed image data (step S502).

After that, processing proceeds to step S503. In step S503, image takingis carried out with the setting of the exposure mentioned above. Theimage processing section 110 carries out image processing including thewhite balance processing using the white balance coefficients that aredetermined by the white balance control processing, and after that,allows the result of the image processing to be displayed on the EVFscreen. Then, in step S504, pixel data within the color capture frame802 is obtained from an image that is used for display (an image that isstored in the image display memory 113 or the memory 114) on the EVFscreen in step S503. In step S410, a specified color is determined byusing the pixel data obtained through the processing operations carriedout in steps S501 through S504.

As described above, in the second embodiment, one difference from thefirst embodiment is that the exposure processing and the white balanceprocessing are carried out at the time of determination of a specifiedcolor. That is, in the first embodiment, when the left button of thecross-shaped button 209 is pressed, pixel values within the colorcapture frame 802 are averaged by using the image data that is currentlydisplayed on the EVF screen to determine a specified color. On the otherhand, in the second embodiment, when the left button of the cross-shapedbutton 209 is pressed, the exposure control processing and the whitebalance control processing are carried out again, and the average valueof pixel values within the color capture frame 802 is computed andobtained by using an image that have been photographed and undergone theimage processing in accordance with the settings set by the exposurecontrol processing and the white balance control processing.

Further, in the second embodiment, in determining a specified color, animage photographed after the exposure is adjusted and controlled to beat an appropriate level is used. Therefore, in the second embodiment,the specified color can be determined by using an image that is capturedwith an appropriate brightness, and further, the user can capture thespecified color with higher accuracy.

In addition, in the second embodiment, in determining a specified color,parameters of the white balance processing section 301 are appropriatelyset before an image that is processed by the image processing section110 is used. It is therefore possible to capture a specified color in anappropriate color level, and thus the user can capture the specifiedcolor with higher accuracy.

Third Embodiment

Hereinbelow, a description is provided of a third embodiment of thepresent invention. As described above, the image pickup apparatus hastwo different modes, namely, an ordinary photographing mode and a colorleaving mode in which a specified color is set and color signals of thecolors other than the specified color are eliminated. In the thirdembodiment, the interval for performing the exposure control processingand the interval for performing the white balance processing are set tobe different between the color leaving mode and the ordinaryphotographing mode. That is, the interval for performing the exposurecontrol processing and the interval for performing the white balanceprocessing in the case of the color leaving mode are set to be shorterthan those in the case of the ordinary photographing mode.

Note that the processing in the ordinary photographing mode isequivalent to the processing shown in FIG. 4, except for the portion ofthe processing related to color conversion, namely, except for theprocessing performed in steps S408 through S411. That is, when theordinary photographing mode is set, an exposure is determined first.When the exposure setting is determined, an aperture value and a shutterspeed are determined on the basis of the determined exposure. Then, thedetermined aperture value and the determined shutter speed are set as anaperture value and a shutter speed, respectively, in the image pickupapparatus of this embodiment. The exposure control carried out in thiscase is the exposure control of an image to be displayed on the EVFscreen. In this regard, a frequent change of the exposure control causesflickering of an image displayed on the screen, and in order to preventthis, a time constant is set with respect to the exposure controlprocessing. For example, the time constant is set so that the exposurecontrol processing is carried out, for example, once in every twoseconds (steps S402 and S403). Next, the white balance processing iscarried out on the basis of data of an image photographed in accordancewith the aperture value and the shutter speed that have been set. Justas in the case of the exposure control processing, a frequent change ofthe white balance control causes flickering of an image displayed on thescreen. In order to prevent this, a white balance coefficient forperforming white balance processing is computed, for example, once inevery five seconds, and thus the white balance coefficient for use inimage processing is updated (steps S404 and S405). Next, image taking iscarried out, and the photographed image is displayed on the EVF screenafter being subjected to image processing by the image processingsection 110 (steps S406 and S407). When the user presses the shutterbutton, the image pickup apparatus of this embodiment carries outphotographing operations (steps S412 and S413). If the user does notpress the shutter button, processing returns to step S402.

Meanwhile, in the color leaving mode, it is necessary to capture aspecified color with a higher color reproduction capability than in thecase of the ordinary photographing mode, and accordingly, a timeconstant for the exposure control processing as in steps S402 and S403in FIG. 4 is set to be shorter than the time constant in the case of theordinary photographing mode. In addition, in the white balance controlprocessing in steps S404 and S405 also, just as in the case of theexposure control processing, a time constant of the white balancecontrol processing is set so that the white balance control processingis carried out at a time interval shorter than the time interval in thecase of the ordinary photographing mode, namely, once in every onesecond, for example. In addition, the setting is performed so that awhite balance coefficient is computed and the white balance coefficientis updated. Although the amount of flickering of an image displayed onthe EVF screen caused in the color leaving mode thereby becomes largerthan in the case of the ordinary photographing mode, the exposure andthe white balance are always set to be in an appropriate level, andtherefore, it is possible to set a specified color with a higheraccuracy.

In order to implement the operations according to this embodiment asdescribed above, the system control section 109 carries out processingas shown in FIG. 11. In the processing shown in FIG. 11, first, in stepS601, it is determined whether the photographing mode is shifted to thecolor leaving mode or not. If it is determined that the photographingmode is shifted to the color leaving mode, processing proceeds to stepS602. In step S602, the interval for performing the exposure control isset to be Ma times per unit time (in the example as described above,once in every one second). Further, processing proceeds to step S603,where the interval for performing the white balance control processingis set to be Mw times per unit time (in the example as described above,once in every one second). On the other hand, if the color leaving modeends, that is, if the photographing mode returns to the ordinaryphotographing mode, for example, processing proceeds from step S604 tostep S605. In step S605, the interval for performing the exposurecontrol is set to be Na times per unit time (in the example as describedabove, ½ time in every one second). Further, processing proceeds to stepS606, where the interval for performing the white balance controlprocessing is set to be Nw times per unit time (in the example asdescribed above, ⅙ time in every one second).

As described above, in a case where the number of times of performanceof the exposure control operation during the display of an image on theEVF screen in the case of the ordinary photographing mode per unit timeis Na and the number of times of performance of the exposure controloperation during the display of an image on the EVF screen in the caseof the color leaving mode per unit time is Ma, a relation of Na<Ma canbe set. According to this setting, an image at the time of colorcapturing thereof in the color leaving mode can be previewed with a moreappropriate exposure. Thus, the user can more easily capture a specifiedcolor with a higher accuracy.

That is, it is possible to capture a specified color with an appropriateexposure corresponding to the variance or change in the place where animage is taken, the time at which an image is taken, and anenvironmental illuminance under which an image is taken, by making thetime taken to perform the exposure control operation in color capturingin the color leaving mode shorter than the time taken to perform theexposure control operation in the ordinary photographing mode.

In addition, in a case where the number of times of performance of thewhite balance control operation during the display of an image on theEVF screen in the case of the ordinary photographing mode per unit timeis Nw and the number of times of performance of the white balancecontrol operation during the display of an image on the EVF screen incolor capturing in the case of the color leaving mode per unit time isMw, a relation of Nw<Mw can be set. According to this setting, an imageat the time of color capturing thereof can be previewed with a moreappropriate exposure. Thus, the user can more easily capture a specifiedcolor with a higher accuracy.

Note that in the second embodiment as described above, an AE photometryof the exposure control at the time of capturing an image used fordetermining the color leaving mode is performed by the same evaluationphotometry using the whole part of the image as in the case of theordinary photographing mode. However, in a case where the regions otherthan the inside of the color capture frame of an image are extremelydark or bright, it is sometimes the case with the evaluation photometryusing the whole part of the image that a specified color within thecolor capture frame cannot be captured with an appropriate brightness.In order to prevent this from occurring, in carrying out the exposurecontrol processing of step S501 shown in FIG. 10, the exposure controlmay be performed by changing the type of photometry to a spot photometryso that the luminance within the color capture frame is at anappropriate level. With this configuration, the luminance within thecolor capture frame 802 can be more appropriately controlled.

Note that in the second embodiment, it has been explained that theexposure control processing and the white balance control processing arecarried out at the time of capturing a specified color in the case ofthe color leaving mode. However, in this regard, automatic focusprocessing may also be performed at the time of capturing a specifiedcolor in the same way. However, in capturing a specified color in thecase of the color leaving mode, the accuracy of the automatic focusingmay be lowered in order to carry out the color capturing at a highspeed. The lowering of the accuracy of the automatic focusing may beeffected by setting a driving step width of a focusing lens forobtaining an evaluation signal for determining a focusing position to begreater than a driving step width in the case of the ordinaryphotographing mode.

Further, in a case where the exposure and the white balance at the timeof capturing of a specified color and the exposure and the white balanceat the time of main exposure photographing are mutually different by alarge amount, there is a case where an image with a specified color leftas desired by the user cannot be taken. In this regard, as shown in FIG.14, photographing operations in step S406 and thereafter may be carriedout in a state in which an auto-exposure (AE) lock switch is operated(step S704) and a white balance (WB) lock switch is operated (stepS707).

Further, with regard to the WB lock, in a case where a white-paper whitebalance is set that uses a white balance control value obtained bypreviously capturing an achromatic subject in a color of white and thelike (a white image) and by performing computation based on the capturedimage of the achromatic subject, a specified color may be captured whilelocking (fixing) white balance with the white-paper white balancecontrol value. In addition, exposure compensation may be prohibited sothat the exposure at the time of capturing a specified color and theexposure at the time of main exposure photographing are not mutuallydifferent.

As described above, according to exemplary embodiments of the presentinvention, a specified color can be determined by actually capturing thespecified color with a digital camera. Thereby, it is possible, inperforming an image pickup operation, to readily obtain an imagesubjected to a special effect in which a color specified by the user isleft and color components of the colors other than the specified colorare converted into specific color component values.

Other Embodiments

Note that in the embodiments of the present invention as describedabove, parameters for converting color component values of the colorsother than the specified color into values indicating the achromaticcolor (i.e., gray scale) are determined. However, the color componentvalues of the colors other than the specified color are not limited tobe converted into the achromatic color but may be converted into colorcomponent values of another chromatic color. That is, the luminancevalue Y is not changed to sustain the tone reproduction as it is, andcolors other than the specified color are represented by a specificchromatic color.

Each unit that configures an image pickup apparatus of an exemplaryembodiment of the present invention and each step of an image pickupmethod of an exemplary embodiment of the present invention as describedabove may be implemented by executing a program stored in a RAM, a ROM,and the like of a computer. The present invention includes the programand a computer-readable storage medium on which the program is recorded.

In addition, the present invention may be implemented as a system, anapparatus, a method, a program, a storage medium, or the like, forexample. More specifically, the present invention may be applied to asystem configured by a plurality of devices or may be applied to anapparatus configured by one single device.

Note that the present invention may also be provided by supplying asoftware program that implements the functions of the above embodimentsto a system or an apparatus, directly or from a remote place, and bycausing a computer of the system or the apparatus to read and executethe supplied program code.

Accordingly, the program code that is installed into the computer inorder to achieve the functions and the processing of the presentinvention implements the present invention. That is, the presentinvention includes a computer program for implementing the functions andthe processing of the present invention.

In this case, the program may be of the form of object code, a programexecuted by an interpreter, script data to be supplied to an OS, and thelike, as long as it is provided with a program function.

A recording medium for supplying the program includes, for example, afloppy (registered trademark) disk, a hard disk, an optical disk, amagneto-optical disk, an MO, a CD-ROM, a CD-R, a CD-RW, a magnetic tape,a non-volatile memory card, a ROM, a DVD (DVD-ROM, DVD-R), and the like.

In addition, the above program can also be supplied by connecting to awebsite on the Internet using a browser of a client computer and bydownloading, from the website to a recording medium such as a hard diskand the like, the computer program itself or a compressed file includingan automatic installation function.

Alternatively, the functions of the above embodiments can also beimplemented by dividing program code that configures a program accordingto the present invention into a plurality of files and by downloadingeach divided file from different web pages. That is, a WWW server forallowing a plurality of users to download program files for implementingthe functions and processing of the embodiments configures the presentinvention.

The above program can also be implemented by distributing a storagemedium such as a CD-ROM and the like that stores the program afterencryption thereof, by having a user who is qualified for a prescribedcondition download key information for decoding the encryption from awebsite via the Internet, and by executing and installing in a computerthe encrypted program code using the key information.

In addition, the functions according to the embodiments described aboveare implemented not only by executing the program read by the computer,but also implemented by the processing in which an OS or the likerunning on the computer carries out a part of or the whole of the actualprocessing on the basis of instructions given by the program.

Further, after the program thus obtained is written in a memory withinan image pickup apparatus via a recording medium or in a state where theimage pickup apparatus is connected to a computer, a part of or thewhole of the actual processing is carried out to implement the functionsof the embodiments of the present invention as described above.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures, and functions.

1. An image pickup apparatus comprising: an image pickup unit; an imageprocessing unit configured to process an image signal obtained by theimage pickup unit; a recording unit configured to record the imagesignal that is outputted from the image processing unit; a display unitconfigured to display on an electronic viewfinder screen an image basedon the image signal that is picked up by the image pickup unit andoutputted from the image processing unit; a color determination unitconfigured to determine a color value based on color informationincluded in a predetermined region of an image that is being displayedon the electronic viewfinder screen; a color conversion parameterdetermination unit configured to determine color conversion parametersfor leaving the color value determined by the color determination unitand converting a color component value included in the image signalother than the color value into a predetermined color value; aninputting unit configured to input an instruction for causing the colordetermination unit to start determining the color value; and anexecution unit configured to execute an exposure control toappropriately set an exposure of the image pickup unit when theinstruction is inputted by the inputting unit, wherein the image signalis processed by the image processing unit based on the parametersdetermined by the color conversion parameter determination unit and theprocessed image is displayed by the display unit, and wherein the colordetermination unit does not determine the color value using an imagepicked up by the image pickup unit before the exposure control isexecuted by the execution unit.
 2. The image pickup apparatus accordingto claim 1, further comprising: an image pickup instruction unitconfigured to generate an image pickup instruction; and a storage unitconfigured to, responsive to the image pickup instruction generated bythe image pickup instruction unit, store on a storage medium the imagesignal processed by the image processing unit based on the parameters.3. The image pickup apparatus according to claim 1, wherein the displayunit displays the predetermined region used in the color determinationunit as a frame on the electronic viewfinder screen.
 4. The image pickupapparatus according to claim 3, further comprising: a frame setting unitconfigured to allow a user to set a position and a size of the framecorresponding to the predetermined region.
 5. The image pickup apparatusaccording to claim 1, wherein the display unit displays a patch imagehaving the color value at a predetermined position as well as on theelectronic viewfinder screen.
 6. The image pickup apparatus according toclaim 1, wherein the color conversion parameter determination unitdetermines color conversion parameters for converting the predeterminedcolor value into a value that indicates an achromatic color.
 7. Theimage pickup apparatus according to claim 1, wherein the colorconversion parameter determination unit determines color conversionparameters for converting a U value and a V value of each lattice pointother than a lattice point corresponding to the color value or latticepoints in the vicinity of the color value, among lattice points that areset within a YUV color space, into zero.
 8. An image pickup apparatuscomprising: an image pickup unit; an image processing unit configured toprocess an image signal obtained by the image pickup unit; a recordingunit configured to record the image signal that is outputted from theimage processing unit; a display unit configured to display on anelectronic viewfinder screen an image based on the image signal that ispicked up by the image pickup unit and outputted from the imageprocessing unit; a color determination unit configured to determine acolor value based on color information included in a predeterminedregion of an image that is being displayed on the electronic viewfinderscreen; a color conversion parameter determination unit configured todetermine color conversion parameters for leaving the color valuedetermined by the color determination unit and converting a colorcomponent value included in the image signal other than the color valueinto a predetermined color value; an inputting unit configured to inputan instruction for causing the color determination unit to startdetermining the color value; and an execution unit configured to executean exposure control to appropriately set an exposure of the image pickupunit when the instruction is inputted by the inputting unit, wherein theimage signal is processed by the image processing unit based on theparameters determined by the color conversion parameter determinationunit and the processed image is displayed by the display unit, andwherein the color determination unit determines the color value using animage picked up by the image pickup unit after the exposure control isexecuted by the execution unit.
 9. The image pickup apparatus accordingto claim 8, further comprising: an image pickup instruction unitconfigured to generate an image pickup instruction; and a storage unitconfigured to, responsive to the image pickup instruction generated bythe image pickup instruction unit, store on a storage medium the imagesignal processed by the image processing unit based on the parameters.10. The image pickup apparatus according to claim 8, wherein the displayunit displays the predetermined region used in the color determinationunit as a frame on the electronic viewfinder screen.
 11. The imagepickup apparatus according to claim 10, further comprising: a framesetting unit configured to allow a user to set a position and a size ofthe frame corresponding to the predetermined region.
 12. The imagepickup apparatus according to claim 8, wherein the display unit displaysa patch image having the color value at a predetermined position as wellas on the electronic viewfinder screen.
 13. The image pickup apparatusaccording to claim 8, wherein the color conversion parameterdetermination unit determines color conversion parameters for convertingthe predetermined color value into a value that indicates an achromaticcolor.
 14. The image pickup apparatus according to claim 8, wherein thecolor conversion parameter determination unit determines colorconversion parameters for converting a U value and a V value of eachlattice point other than a lattice point corresponding to the colorvalue or lattice points in the vicinity of the color value, amonglattice points that are set within a YUV color space, into zero.
 15. Theimage pickup apparatus according to claim 8, wherein the colordetermination unit determines the color value using an image outputtedfrom the image processing unit after the parameter for white balanceprocessing is appropriately set by the execution unit.
 16. A method ofcontrolling an image pickup apparatus, the method comprising: an imagepickup step of converting light from a subject into an image signalusing an image pickup element; an image processing step of processingthe image signal obtained in the image pickup step; a display step ofdisplaying on an electronic viewfinder screen an image based on theimage signal that is picked up in the image pickup step and outputted inthe image processing step; a color determination step of determining acolor value based on color information included in a predeterminedregion of an image that is being displayed on the electronic viewfinderscreen; a color conversion parameter determination step of determiningcolor conversion parameters for leaving the color value determined inthe color determination step and converting a color component valueincluded in the image signal other than the determined color value intoa predetermined color value; a control step of processing the imagesignal in the image processing step based on the parameters determinedin the color conversion parameter determination step and of displayingthe processed image signal in the display step; an inputting step ofinputting an instruction for starting determining the color value in thecolor determination step; and an execution step of executing an exposurecontrol to appropriately set an exposure of the image pickup elementwhen the instruction is inputted in the inputting step, wherein in thecolor determination step, the color value using an image picked up bythe image pickup unit before the exposure control is executed by theexecution unit is not determined.
 17. A storage medium storing a controlprogram, which when loaded into a computer causes the computer toexecute the method according to claim
 16. 18. A method of controlling animage pickup apparatus, the method comprising: an image pickup step ofconverting light from a subject into an image signal using an imagepickup element; an image processing step of processing the image signalobtained in the image pickup step; a display step of displaying on anelectronic viewfinder screen an image based on the image signal that ispicked up in the image pickup step and outputted in the image processingstep; a color determination step of determining a color value based oncolor information included in a predetermined region of an image that isbeing displayed on the electronic viewfinder screen; a color conversionparameter determination step of determining color conversion parametersfor leaving the color value determined in the color determination stepand converting a color component value included in the image signalother than the determined color value into a predetermined color value;a control step of processing the image signal in the image processingstep based on the parameters determined in the color conversionparameter determination step and of displaying the processed imagesignal in the display step; an inputting step of inputting aninstruction for starting determining the color value in the colordetermination step; and an execution step of executing an exposurecontrol to appropriately set an exposure of the image pickup elementwhen the instruction is inputted in the inputting step, wherein in thecolor determination step, the color value using an image picked up bythe image pickup unit after the exposure control is executed by theexecution unit is determined.
 19. A storage medium storing a controlprogram, which when loaded into a computer causes the computer toexecute the method according to claim 18.